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Using Experimental Methods to Investigate Discriminatory Tendencies: A Lesson Report 

Using dictator games in experimental analysis, this lesson report demonstrates the process and results of a 
postgraduate class project in which university students were instructed to scientifically investigate and explore one of 
German society's most hotly-contested issues: the level of discriminatory tendencies of non-Muslims towards 
Muslims. The results of this class project show little or no discriminatory tendencies toward Muslims. Instead, the 
university students under our investigation tended to act favorably, or at the very least, fairly toward Muslims. We 
expect that this lesson report can demonstrate how a postgraduate course can be conducted in an innovative way, 
empowering students to collect primary data and finishing a small scientific project during the span of a semester. 

Dieser Bericht stellt die Prozesse und Ergebnisse einer experimentellen Studie eines Postgraduierten-Kurses der 
Universitat Greifswald dar. Wir zeigen, wie Studierende das Thema Migration innovativ untersuchen konnen, indem 
sie neben theoretischen Lerninhalten an experimentelle Methoden sowie die Primarforschung herangefuhrt werden. 
Anhand eines Diktatorspiels wurden die teilnehmenden Studenten angeleitet, ein in der Offentlichkeit kontrovers 
debattiertes Thema wissenschaftlich zu untersuchen: Diskriminierende Tendenzen von Nicht-Muslimen gegenuber 
Muslimen. Die Ergebnisse dieser Studie lassen jedoch nicht auf diskriminierende Tendenzen der Studenten schliefSen. 
Stattdessen deuten sie darauf hin, dass sich die Studierenden zumindest fair gegenuber Muslimen verhalten. 
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1 Introduction 

Experimental research methods have been increasingly 
applied across a wide range of social science disciplines, 
from ethnography to political science (Kittel, Morton, 
2012, p. 1; Xiang, Toyota 2013). In addition, several 
comprehensive textbooks have been published on the 
manifold applications of experimental methods in 
political science (Morton, Williams 2010; Mutz 2011), 
which provide overviews about various topics that can be 
studied using experiments. 

Although laboratory experiments are often considered 
the gold standard for establishing causal inference in the 
natural sciences, they have only occasionally been used 
in the social sciences, and as an ideal application, have 
often been regarded as hardly feasible (King et al. 1994, 
125; Morton, Williams, 2010, p. 31). However, next to 
technological developments, due to a rising critique on 
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observational studies and their shortcoming to allow 
causal inferences, the application of experiments has 
been advocated since the late 1990s (Morton, Williams, 
2010, 3), and some see a future for political science as an 
experimental research discipline (Morton, Williams 2010, 
p. 529). A stronger established approach of experiments 
can lead to careful methodological considerations that 
minimize current shortcomings in quantitative political 
science studies (Schrodt 2013) and improve scientific 
outcomes. 

We argue that experimental methods can offer distinct 
advantages over other research strategies, but should 
only be applied after it has been tested and proved to be 
a favorable strategy to investigate a certain question. 
Thus, students need to learn about the advantages and 
disadvantages of experiments, and understand when it is 
appropriate to employ an experimental method. To date, 
experimental methods are rarely taught to postgraduate 
students in political science - almost exclusively at top- 
ranking American universities - and in most cases only if 
their lecturers had previous formal training in 
experimental methods (Myers 2013, p. 13; Morton, 
Williams 2010, p. 22). 

This lesson report aims at providing practical advice on 
how to teach experimental research methods to 
students. It documents the process and results of a 
postgraduate class project taught at the University of 
Greifswald where students were instructed to design an 
experiment to investigate students' discriminatory ten¬ 
dencies of non-Muslims towards Muslims in Germany. 
With a construct known as the dictator game, which is 
most often used in experimental economics, the lecturer 
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led students to measure just how much non-Muslim 
players are willing to donate to a Muslim firm. From 
research design, data collection, data analysis, and report 
writing, students gained hands-on experience in using an 
experimental method to explore the highly-contested 
relations between Muslims and non-Muslims in their 
own society. This report is written by the lecturer of this 
course, interestingly a non-German, and two of the 
participating German students who shared the learning 
experiences. 

The pedagogical purposes of this class project will be 
specified in the next section. The third section of this 
lesson report is a summary of the literature review which 
the lecturer led the students to conduct before 
embarking on their research project. Students were 
asked to review literature on four theoretical models 
used to explain discriminatory behaviors toward 
immigrants, and shed light on how such models can be 
applied to in-group/out-group bias toward Muslims in 
Germany. We also reviewed studies that have used 
experimental analysis to gauge discrimination in ethnic 
and migration studies, and discuss how such methods 
can be applied to the research presented here. In section 
four, we report the concrete research question and 
hypothesis that the lecturer and the students agreed to 
work on after class discussion. In section five, we share 
our experience of brainstorming and finding a valid and 
innovative research method, design, and dataset. In 
section six, we present an analysis of the experimental 
data collected by the students and explore the 
implications of the findings. Finally, we conclude by 
discussing what the lecturer and the students have 
learned from this collaborative learning experience and 
propose recommendations for future class projects. 

2 Pedagogical Design and Purposes 

Prior to taking this module, most students had no 
experience with primary empirical research processes, 
although they had learned theories and research 
methods in other modules and were able to run 
statistical analyses of secondary data. In addition, none 
of the students had been previously involved in experi¬ 
mental methods. 

The lecturer had received previous training of 
experimental methods at the University of Konstanz in 
Germany and had decided to share this method with the 
students at Greifswald. While preparing the seminar, the 
lecturer set two main learning objectives. 

The first was to lead students to survey and review 
current social science literature that explores xenophobic 
attitudes in various countries and in Germany where the 
module is taught. There were discussions on the 
historical background and current development of the 
hotly-debated relations between Muslims and non- 
Muslims in Germany (Rommelspacher 2004; Fetzer, 
Soper, 2005; Flaug et al. 2009). Even Muslims who 
immigrated to Germany years or even decades ago are 
seen as outsiders by German non-Muslims. 


The lecturer encouraged students to gain a compre¬ 
hensive understanding of what has been scientifically 
done to investigate the relations between Muslims and 
non-Muslims in Germany. It is assumed that students are 
informed of the highly contested issues of Muslim vs. 
non-Muslim relations in Germany in the public debate 
(e.g., on TV, in newspapers). Flowever, students might 
not be aware of what social scientists have and can do to 
study this issue. 

The heart of this class project is based on questioning 
non-Muslim Germans living in areas where people have 
little or no social contact with Muslim immigrants to see 
the extent that they exhibit suspicion toward Muslims. 
Existing survey analyses indicate a higher level of 
ethnocentrism in the newly-formed states (neue Lander) 
of the former German Democratic Republic (GDR) than in 
the Federal Republic of Germany, or what used to be 
called West Germany (Alba et al. 2004). 

After the literature review and background study, the 
lecturer led students to discuss the pros and cons of 
different research methods to explore "discriminatory 
tendencies" and asked how they can be applied to 
evaluate non-Muslims' attitudes towards Muslims in 
Germany. In other words, the second objective that the 
lecturer aspired to achieve was methodological. 

Surveys have traditionally been used to investigate 
attitudes between in-groups and out-groups (McConahay 
1982; Kinder, Sanders 1996; Pettigrew 2000; Wasmer, 
Koch 2003). The most commonly noted concern about 
surveys is that, at best, they gauge the stated opinions of 
respondents. However, what people say about their 
beliefs and actions can be very different from what they 
actually think or how they behave in day-to-day life. 

Moreover, in light of the long-standing emphasis on 
egalitarian principles in Western societies, respondents 
might either consciously or unconsciously pay lip service 
to such principles, which may or may not reflect their 
inner-most preferences and sensibilities and is known as 
the principle implementation gap (Kinder, Sanders 1996, 
p. 92-127 & 291-294; Wasmer, Koch 2003, p. 103). 

Experiments thus provide an alternative way to 
measure discriminatory tendencies. Ideally, experiments 
would control every variable. In the current case, that 
would mean that the results (i.e., behaviors induced by 
the experiment) reflect as closely as possible the genuine 
level of discriminatory tendencies of non-Muslim 
German students towards Muslims. In other words, 
unlike surveys, experiments should not merely measure 
what respondents self-report, but what their actual 
beliefs and behaviors are. 

As "learning by doing" is the lecturer's teaching 
philosophy, the lecturer instructed the students to 
conduct a small-scale experiment on their own, linking 
the project to their curiosity of the relationship between 
Muslims and non-Muslims in their immediate 
environment, the University of Greifswald. As part of this 
methodological objective, the lecturer also had to 
prepare students to undertake primary research, which 
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they have not learned in previous courses. For the 
experimental method, students were thus led to discuss 
all emerging issues, such as possible ethical concerns 
that might arise when recruiting participants and using 
forms of deceptions. The class worked together to find 
appropriate ways to meet these challenges. 

In the following section, we present the "ground work" 
that the class was led to conduct before they embarked 
on the empirical investigation. That is, a literature review 
on theories of xenophobic attitudes, a background study 
of Muslims in Germany, as well as a review of studies 
that have been employed in experimental methods to 
investigate discrimination and trust. 

3 Literature Review 

3.1 Theories on Xenophobic Attitudes 

At the outset of this course, students analyzed and 
discussed scientific articles on discrimination and 
xenophobia in class. The students believed that this 
would provide an invaluable basis for all subsequent 
steps of the primary research process. 

In general, there are four models for explaining 
discriminatory attitudes toward immigrants: compe¬ 
tition, contact, frustration, and ideology models. 
Although these models are easily differentiated, they 
share one essential feature: their theoretical under¬ 
pinnings are all based on social group theory. We will 
now explore the basic aspects of this theory before tying 
together what are in essence four types of social group 
theory. 

Social group theory is most often utilized to elucidate 
identity formation. This includes not only personal 
identity but collective identity in regards to regions or 
nations. Its assumptions are based on a common idea - 
the identity-alterity nexus (Guillaume 2007; Roberts 
2007). According to the concept of alterity, an identity 
can only be constructed when the self distinguishes 
between itself and the other: "Who we are is usually 
framed as a response to some 'other' group" (Fligstein 
2009, p. 135). Flence, self-awareness, self-images, and 
identity can only be formed in a fluid process while the 
contrasting other is permanently reflected. This also 
holds true for the formation of social identity because 
"every group needs an outside perspective to be 
recognized" (Neumann 2007, p. 19). Due to the ongoing 
process of social interaction, identities can change over 
time. Flowever, after a given period, collective identities 
settle and remain stable for the most part. This appears 
to be true to such an extent that they can almost be 
regarded as social facts (Risse 2010, 29). 

A collective identity can be exclusive or inclusive. 
Group identities can coexist with one another as multiple 
identities. Every collective identity establishes an 
imagined community. This community is based on "the 
idea that a group of people accept a fundamental and 
consequential sameness that causes them to feel 
solidarity among themselves" (Fligstein 2009, 135). This 
includes a set of rules and norms that define 


membership in the group and frame a collective world 
view. 

The psychology of group membership is commonly 
applied to memberships in "large-scale social categories 
like nationality, class, sex, race or religion" (Turner 1982, 
p. 22). A dichotomous distinction is made between the 
in-group and the out-group. This divide is first and 
foremost a neutral operation that does not necessarily 
imply hostility toward the out-group (Brewer 1999). 
Flowever, in-group-favoritism occurs consistently (Risse 
2009, p. 152). 

Many types of experiments, such as field and student 
experiments, have been conducted and have subse¬ 
quently shaped the theoretical assumptions of social 
group psychology. By using economic incentives, 
experiments have shown that participants tend to favor 
their own group, prefer cooperating with other 
participants from their own group, and are more likely to 
be suspicious of members of the perceived out-group 
(Ruffle, Sosis 2006, p. 147). 

The in-group/out-group distinction is often used to 
explain prejudices against ethnic or religious minorities 
because in modern nations, they typically form the out¬ 
group. Within a nation, a perceived cultural threat to the 
dominant nationality can amplify prejudices into 
discriminatory and xenophobic attitudes (McLaren 2002, 
p. 554). This fear is assumed to threaten either national 
resources or the dominant cultural way of life (Bobo 
1983). Such a perceived threat strengthens the feeling of 
belonging within the dominant group and increases 
hostility toward the minority. These notions of social 
group theory are closely tied to the competition model. 
In situations of a perceived economic threat, induced 
perhaps through high unemployment or economic crisis, 
attitudes toward immigrants become more hostile. It has 
been argued that this perceived threat is felt more 
strongly by low-skilled members of the working class 
than those with higher socioeconomic status, as the 
former are more likely to compete directly on the labor 
market against immigrants (Alba et al., 2004). 

Alternatively, the contact model argues that the 
number of contacts people have with immigrants is a 
crucial indicator of discrimination or xenophobia (Allport 
1954). In this case, a higher degree of contact is 
associated with fewer xenophobic attitudes. Conversely, 
people who do not share a common living environment 
or any other type of contact with a particular group of 
immigrants are more likely to develop discriminatory 
attitudes toward them (Pettigrew, Tropp 2006). 

A third model, the frustration model, states that given 
particular social and economic situations, a certain level 
of frustration among members of the dominant social 
group can be expected to lead to a rise in xenophobic 
attitudes toward minority groups. As a social out-group, 
this particular minority is blamed for the unfavorable 
state of the country or the state of social deprivation felt 
by people in the dominant group. Flence, the minority is 
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functionally cast in the role of scapegoat for the 
dominant social group. 

Finally, the ideology model is strongly based on the in- 
group/out-group nexus, whereby the crucial difference 
between the in-group and the out-group is constructed 
around their ideologies. The ideology model is often used 
by populist right-wing politicians who proclaim the 
general beliefs and values of the in-group, and insist that 
the out-group are incompatible and that social 
integration is doomed to fail. The caveat of the ideology 
model is that it over-emphasizes the differences 
between the in-group and the out-group, negating the 
fact that despite their differences, there can still be 
common characteristics. 

3.2 Muslims in Germany 

There are a variety of immigrant groups in Germany, 
such as guest workers ( Gastarbeiter ), third-world 
refugees, and other ethnic Germans ( Aussiedler). The 
lecturer had planned to lead the students to focus on 
attitudes toward Muslim immigrants in their project 
because the debate in recent years about their 
integration into German society far outstrips any similar 
public debate about other immigrant groups, which has 
been fuelled by populist views such as Sarrazin's (2010). 

We should note, however, that it is often unclear to 
whom the phrase 'Muslim immigrants' applies. For this 
study, the class decided to utilize a broad definition that 
includes all immigrants who are avowed members of the 
Islamic religion, without any reference to the strength of 
their commitment toward their religion. They may have a 
Turkish or Arabic background, they may have arrived as 
guest workers or refugees, and they may be second or 
third generation immigrants. 

According to the German Federal Office for Migration 
and Refugees (Bundesamt fur Migration und Fluchtlinge), 
there are approximately 3.8-4.3 million Muslims in 
Germany (Flaug et al. 2009, p. 11). That makes Islam the 
third largest religion in the country. The variation in 
concentration of the Muslim population between the old 
West German states and the newly formed East German 
states is dramatic: 98.4 percent of Muslims in Germany 
reside in the Western states, while only 1.6 percent 
reside in the newly-formed states of the East. 

Muslim integration into German society is analyzed in 
terms of structural, cultural, and social integration, in 
addition to a number of other dimensions. There is 
ample literature on this theme (for example Hein 2012). 
Most of these findings can be put under the rubric of the 
contact model. Empirical studies attempt to analyze how 
strong the contact of Muslim immigrants is to other 
social groups in light of self-segregation in a so-called 
'parallel society', which is perceived as the greatest 
obstacle to successful integration. 

Furthermore, the competition and frustration models 
have gained wider acceptance by virtue of recent 
findings that show right-wing attitudes increasing since 
the start of the recent economic crisis. Right-wing and 


nationalist attitudes are most often connected to in- 
group-favoritism and out-group-hostility. Flence, these 
findings have given rise to concerns that economic crisis 
exacerbates xenophobic attitudes toward minorities like 
Muslim immigrants (Decker, Brahler 2010, p. 95). 

Flowever, the most widely supported explanatory 
model regarding the integration debate is the ideology 
model. Populist right-wing politicians frequently empha¬ 
size and construct differences between Muslim 
immigrants and the rest of the German population. They 
stigmatize and generalize Muslims as being religious 
fundamentalists (Rohe 2011, p. 23). By doing so, they 
oversimplify the heterogeneity of Muslim immigrants 
and foster prejudice. Furthermore, superficialities, such 
as headscarves, are depicted as Muslim symbols with a 
political dimension representing anti-democratic values 
($eker 2011, p. 16). 

As the foregoing demonstrates, there is theoretical and 
empirical support for all four explanatory models. All 
point to the fact that discrimination toward Muslim 
immigrants is likely to occur in Germany. The following 
sub-section shows how experiments can be used to study 
these topics. 

3.3 Experimental Methods in the Study of In- 
Group/Out-Group Bias 

The theoretical session of lectures provided an overview 
of various experimental methods (e.g., field, laboratory, 
and natural), their specific designs (e.g., economic games 
and Solomon four-group designs), and their applications 
to human behavior in social science disciplines (e.g., 
behavioral economics, social psychology). The lecturer 
led the students to design an experiment around the use 
of dictator games, and referred the students to previous 
such experiments. For instance, scholars have already 
used experimentation to study ethnic discrimination in 
Israel (Fershtman, Gneezy 2001), racial discrimination in 
South Africa (Burns 2006), and linguistic segmentation in 
Belgium (Fershtman et al. 2005), but few have used 
experimentation to explore ethnic discrimination in 
Germany (Klink, Wagner, 1999). To the best of the 
lecturer's knowledge, no method of experimentation 
specifically dedicated to investigating the schism 
between Muslims and non-Muslims in Germany has been 
utilized. Flowever, such an approach could be fruitful for 
exploring these issues. As previously noted, experiments 
can assist us in isolating the genuine beliefs and behavior 
of subjects. To a certain degree, this might remedy flaws 
in surveys that only consist of the voluntary responses of 
interviewees, which at best gauge the respondents' 
opinions. Indeed, even during an experiment, 
participants may not behave according to their inner 
preferences, but to socially-desirable norms, particularly 
when they know they are being observed. This concern 
cannot be completely dispelled. Flowever, while there is 
no foolproof social science approach to sifting out a 
respondent's inner and outer reactions, an experimental 
approach can strengthen the validity of findings. It is in 
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this manner that the lecturer expects the experimental 
approach to speak to current survey analyses. 

4 Research Question and Hypothesis 

After the students finished the literature review, they 
were led to choose a research question and propose 
hypotheses. The class decided to have a research 
question that focuses on student populations: Do non- 
Muslim students demonstrate discriminatory tendencies 
toward Muslims in the newly-formed states of East 
Germany? 

It is posited that non-Muslims in East Germany will 
show discriminatory tendencies toward Muslim 
immigrants because first of all, as various studies have 
shown, individuals tend to favor in-group members over 
out-group ones. This is true even when group affiliation 
is 'artificially' created, or even when the two groups are 
not in competition (Sherif et al. 1988). 1 Beliefs in Islam 
and the backgrounds of the migrants initially 
differentiated Muslims from non-Muslims in Germany. 
Historically, non-Muslim Germans have tended to 
perceive Muslims as members of the out-group. 

According to contact theory, in areas where there are 
fewer social contacts with Muslim immigrants, such as in 
East Germany where the university is located, 
discrimination exhibited by non-Muslims toward Muslims 
is expected to be higher. 

However, one has to keep in mind that university 
towns usually attract students from afar and in the case 
of Greifswald, this includes students who originate from 
multicultural cities such as Berlin and Hamburg. Hence, 
the student's background is crucial to determine whether 
the contact hypothesis can be applied. 

The frustration and competition models also support 
the hypothesis. As Alba et al. (2004) anticipate that a dire 
economic situation in the East (compared to the West) 
could have been cause for a higher level of 
ethnocentrism in the newly-formed German states, 
although other demographic variables (e.g., age) cannot 
be ruled out. Following the competition model, highly 
educated and socioeconomically well people are less 
likely to show discriminatory tendencies. In summation, 
the class posits that non-Muslims will show suspicion 
toward Muslims and this applies to students, even 
though students are less likely than other social groups 
to show discriminatory tendencies. 

5 Methods and Data 

5.1 Solomon Four-Group Design 

To explore whether non-Muslims treat Muslims 
differently than other groups, we used the dictator 
game. The game involves non-Muslim players who split a 
fixed amount of money, in our case two euros (10 x 20- 
cent euro coins), between her- or himself and a fictional 
Muslim partner who is a 'passive' player without any 
decision-making role. If the amount allotted to the 
passive player is affected by her or his Muslim immigrant 


background, this would be an indication of the presence 
of discrimination (Fershtman, Gneezy, 2001). 

To better capture the effect of discrimination against or 
in favor of Muslims, we used a Solomon four-group 
design, consisting of two experimental groups and two 
control groups. The first experimental group included a 
pre-test and a post-test of the dictator game (Table 1). In 
the pre-test, a non-Muslim was asked to decide how 
much she or he would be willing to donate to a 
businessman who was in need of capital to set up a firm 
in Hamburg. 


Table 1: The Solomon four-group experiment 


EG-l 

Random 

assignment 

Pre¬ 

test 

Receive 
experimental 
treatment: briefing 

Post¬ 

test 

CG-1 

Random 
assignment * 

Pre- 
" test 


Post¬ 

test 

EG-2 

Random 

assignment 


Receive 
experimental 
treatment: briefing 

Post¬ 

test 

CG-2 

Random 

assignment 



Post¬ 

test 


Source: Adopted from Carlson and Hyde (2003, 284) 


After the pre-test, the non-Muslim player received an 
experimental treatment: a fact sheet with details about 
the entrepreneur and his business plan. In this fact sheet, 
the entrepreneur is characterized by an obviously Arabic 
name (i.e., Kamran Althani), a picture of an Arabic 
businessman, and the fact that he holds a degree in 
industrial engineering. In addition, the planned firm was 
described as an import-export company marketing fair¬ 
trade coffee products. 

After the briefing (i.e., treatment), the non-Muslim 
player is invited to the post-test game. The same 
procedure that occurred in the pre-test game is 
repeated. The player decides how much she or he is 
willing to offer the passive player. The suggested 
monetary split is then carried out. Up to now, the entire 
procedure constitutes experimental group 1. Next is the 
first control group, conducted in the same manner as the 
experimental group 1, except that the active player 
receives no data on the passive player (i.e., no 
experimental treatment) (Table 1). 

Both groups constitute a classic experimental design in 
which both the experimental and the control groups are 
pre-tested. The weakness of the classic design is that the 
players in the experimental group might be sensitized to 
the experimental treatments they receive. In other 
words, if they are not pre-tested, they could pay closer 
attention to the briefing data. For the purposes of 
controlling this interactive effect between the pre-test 
and the experimental treatment, the lecturer suggested 
to the class to use a Solomon four-group design. In 
essence, the Solomon four-group design adds a second 
experimental group that also receives the treatment, but 
is not pre-tested. Accordingly, in this second experi¬ 
mental group, there is no interaction between the pre¬ 
test and the experimental treatment. Finally, a second 
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control group is added that is neither pre-tested nor 
treated. This group is only post-tested to take account of 
the potential interactive effect between the control 
group pre-test and the control group post-test (Table 1). 
Overall, the Solomon four-group design adds two 
additional groups to boost the internal validity of the 
experiment. 

It can be assumed that if the participant has a tendency 
to discriminate against Muslims, she or he would 
dispense less money in the post-test than the pre-test. 
The treatment in the experimental groups should result 
in a larger difference between pre- and post-test 
donations compared to the control group. Lastly, the 
average amount invested in the post-test game of the 
second experimental group should be less than that 
invested in the related control group. 

5.2 Operation 

The experiment was held at the University of Greifswald. 
Greifswald is located in the East German state of 
Mecklenburg-West Pomerania (Mecklenburg 
Vorpommem), the population of which is only 0.1 
percent Muslim (Haug et al. 2009, p. 107). The possibility 
of playing the game with people on the street had been 
discussed, but was considered too time-consuming, 
labor- and cost-intensive, and unlikely to result in a 
representative sample. Hence, the class chose to play the 
dictator games with students at the University of 
Greifswald. The lecturer proposed a possible 
experimental design based on a dictator game set in a 
Solomon four-group design. The students were 
convinced by its clear advantage - the possibility to test 
the internal validity - and in a second step, were then 
encouraged to further plan, conduct and evaluate the 
experiment as a group effort. Each student contributed 
to different parts of the project such as the stimulus and 
questionnaire design or analysis, while all students 
participated in the data collection and analysis. The 
convenience sample of this study represents a relatively 
homogeneous group with certain characteristics (e.g., 
highly educated and skilled laborer), and the students 
hoped that they could draw inferences from their 
specific sample population (Kam et al. 2007, p. 420). 

We recruited participants using the University's e-mail 
system which allowed us to send an invitation to all 
students of the University. In addition, we advertised the 
project by talking to students, for example, on their way 
to the canteen. We decided to apply only a very light 
form of deception of our research interest by advertising 
that a postgraduate course wants to undertake a political 
science study and searches for participants. We offered 
participants an incentive that they could win one of three 
Amazon vouchers to the value of 20 euros. A group of 
students designed the recruitment letter that informed 
the participants about the context, aim, and duration of 
the study. Each participant's informed consent was 
ensured prior to the experiment. 


In this process, students perceived the design of the 
stimulus as the largest challenge. They debated 
intensively about the information they wanted to share 
with the participants, and opted for information about 
the company and gave the entrepreneur a specific high- 
skilled background to avoid typical stereotypes about 
low-skilled immigrants. Hence, they tried to measure 
latent discriminatory attitudes towards Muslims. 

Due to financial and time limitations, a pilot study was 
not feasible. However, students undertook a small test 
run with a limited number of participants prior to the 
experiment in order to evaluate whether the 
experimental design needed adjustment (e.g., whether 
the participants understood the instructions). This pre- 
experimental training was vital for students to learn how 
to behave coherently during the experiment, and helped 
to minimize any 'interviewer effect' in the actual 
experiment. After the experiment, participants were 
asked to fill out a questionnaire that gauged their 
backgrounds. In the end, the students were able to 
gather a dataset that included a total of 151 participants. 

The dataset contains information about the amount of 
money donated in the pre- and post-tests, as well as data 
about gender, age, education, residence, religion, and 
the participants' views on fair-trade and their preference 
for coffee to control for the use of a fictitious coffee 
company. Overall, the dataset contains 17 variables. A 
group of students entered the data into Statistical 
Product and Service Solutions (SPSS), double-checked 
their entries, created a codebook, and distributed the 
dataset and codebook to the other students. Afterwards, 
students worked together to run an analysis and practice 
interpreting the results. In the end, each submitted an 
individual research report. 

5.3 Debates about Limitations of the Experimental 
Design 

In this module, students were also led to debate over the 
potential limits of their research design, sampling, and 
method. Sampling was particularly contested among the 
students. In the end, students agreed that conducting an 
experiment with students at a university in East Germany 
can be understood as 'theoretical sampling'. This implies 
that this social group is least likely to show dis¬ 
criminatory attitudes toward Muslims. Among all the 
federal states, the absolute number of foreigners who 
live in Mecklenburg-West Pomerania is the smallest by 
far, with 30,068 in 2010 according to the Federal 
Statistical Office (Bundesamt fur Statistik) (2013a). Data 
on the number of people with immigrant backgrounds is 
only given as an aggregate for the newly-formed states 
[neue Lander) of the former GDR, which is dramatically 
lower than the number of foreigners and citizens with 
immigration backgrounds in the 'old states' (Federal 
Statistical Office 2013b). 

Hence, contact with Muslim immigrants might be 
limited in a city like Greifswald, but students at the 
University of Greifswald have diverse origins from across 
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Germany (see Table 2). This means that the contact 
hypothesis does not necessarily apply to them. 
Moreover, it is logical that students with relatively higher 
levels of education would be the least likely societal 
group to develop discriminatory attitudes. If a lack of 
trust toward Muslim immigrants is found among 
students in East Germany, then it is very likely that we 
will find even higher levels of suspicion in other East 
German social groups. 

Numerous studies have been based on student 
samples within the social sciences, especially those 
based on psychological research (King et al. 1994, p. 125; 
Henrich 2010). Their predominant goal is to establish 
internal validity; therefore, external validity is only of 
secondary concern to them. However, student 
convenience samples can pose a threat to the internal 
validity because students have very different 
characteristics than the average person (Hooghe et al. 
2010, 85). For instance, experiments in international 
relations showed in comparison to military elites, 
students apply different decision making strategies 
(Mintz et al. 2006, 765). Therefore, concerns about the 
internal validity are addressed prior to the limited 
external validity, which is often regarded as the 
weakness of experiments (McDermott 2002, p. 334). 

Because student samples are highly contested, the 
research question and objective are crucial for 
determining whether students are an appropriate 
sampling base (Kam et al. 2007, p. 416). According to 
Kam et al. (2007), three cases exist in which student 
samples are appropriate: first, students are the 
underlying population; second, no reason indicates that 
students differ from non-students regarding the subject 
matter; and third, student samples can provide a critical 
test for the research hypothesis (Kam et al. 2007, 420- 
421). The latter reason is the one on which we build the 
case for students as a 'theoretical sample'. 

Besides theoretical implications, a student sample 
offers another advantage over other convenient sample 
populations - it facilitates experimental realism 
(Druckman, Kam 2011, p. 51). In contrast to mundane 
realism, experimental realism is necessary to generate a 
situation in which participants show their true intentions 
because they act without concerns of social desirability. 
This is particularly relevant for economic games such as 
our dictator game in which monetary transfers are made. 
For students, experimental realism can be generated by 
offering significantly lower amounts of money compared 
to other social groups (Druckman, Kam 2011, p. 51; 
Guala 2005, 33-34; Friedman, Sunder 1994, p. 39-40). 
Hence, the students decided to apply the idea of 
theoretical sampling to our study. 

It is also worth noting that the class was constrained by 
lack of finances. Thus, it was only possible to use a 
convenience sample. Discussing the issues around a 
suitable sampling frame, students learned that 
drawbacks are inherent in every research method, and 
even "experiments are not a panacea for all 



methodological concerns" (McDermott 2002, p. 340). 
Therefore, students were advised to carefully reflect 
possible threats to internal and external validity, when 
designing an experiment. 2 One way to achieve a higher 
level of external validity could be obtained by cross- 
validations had we the resources. We will mention this 
point again at the end of this lesson report. 

The class spent a significant amount of time debating 
the inferences that could be obtained from our student 
sample and analyzing feasible alternatives, such as a 
convenience sample with the inhabitants of the city, 
Greifswald, while bearing the impact on other biases in 
mind. Playing the game on the street could also 
introduce a selection bias, because the working 
population might be less likely to be found on the streets 
during the day. Hence, it is extremely difficult to obtain a 
sample which is representative for the whole population 
when faced with limited resources. However, it remains 
interesting to lead students to explore how residents in a 
city of few Muslims would react to and treat a Muslim in 
the experiment, particularly in comparison to a student 
sample as we theorized the possibility of a hete¬ 
rogeneous treatment effect. The outcome could be very 
different from the behavior of students, as we suspected 
a higher impact of the contact and frustration 
hypotheses. Hence, our findings are in a sense likely to 
underestimate the impact of discriminatory tendencies 
that would occur in the general population. Neverthe¬ 
less, non-responsive-rates can be high when recruiting 
non-student participants for experimental studies. 

After considering and debating the aforementioned 
issues, the students decided that they believed in the 
value of a student sample and opted unanimously to 
proceed to use the student sample to test their 
hypotheses (Kam et al. 2007, p. 420; Flyvbjerg 2006, p. 
230). 


Table 2: Greifswald student population in Summer 2010, 
when the experiment was conducted 


Categories 

Numbers 

Percentages 

German students from new states 

7129 

61.14 

German students from old states 

2871 

24.62 

German students from Berlin 

1091 

9.36 

Foreign students 

569 

4.88 

Total number 

11660 

100 


Source: www.uni-greifswald.de/informieren/ 
zahlen/studierende.html (accessed 4 November 2013) 


6 Analysis and Discussion 

6.1 Descriptive Analysis of the Participants 

Eighty-two participants (54.3 percent) were female, 
whereas 69 participants (45.7) percent were male. The 
vast majority (140 participants, or 92.7 percent) reported 
Greifswald as their current residence. About 143 
participants (94.7 percent) were born in Germany and 
145 participants (96 percent) were German citizens, 
while five (3.3 percent) were foreigners and one (0.7 
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percent) had dual citizenship. In addition, only 12 
participants (7.9 percent) had an immigrant background. 

The low percentage of students with immigrant 
backgrounds reflects first of all, that students with 
immigrant backgrounds are underrepresented at the 
University. Second, fewer people with immigrant 
backgrounds live in the newly-formed German states. 
Forty three participants (28 percent) reported that they 
were from West Germany, while 97 (64.2 percent) grew 
up in East Germany. Moreover, the majority (87 
participants, 57.6 percent) had no religious affiliation, 
followed by Evangelical Christian students which 
comprised almost one-third (47 participants, 31.1 
percent) and eight participants (5.3 percent) who were 
Catholic. Muslims, Orthodox Christians, and other 
religious affiliations were very poorly represented, 
making up only 1.3 to 2 percent of the participants. The 
high percentage of atheists is typical for this newly- 
formed German state, and indicates the enduring 
influence of the communist GDR. A cross tabulation of 
religion and the German state of origin suggests that all 
atheist students are from the neue Lander. 

Most of the students were unfamiliar with taking part 
in empirical studies, with 135 participants responding 
that they had never participated in such a procedure 
(89.4 percent). Therefore, most participants were less 
likely to strategically contaminate our experimental 
result. 

Furthermore, a number of descriptive statistics were 
assessed to account for the design of the treatment. 
About one-fifth (29 participants, 19 percent) opposed 
fair-trade, 6 participants (4 percent) had no opinion on 
fair-trade; while the vast majority favored fair-trade (116 
participants, 77 percent). In addition, 53 participants 
(35.1 percent) disliked coffee, whereas 98 participants 
(64.9 percent) liked coffee. We will discuss whether 
these variables affected the outcome of our experiment 
in the next two sub-sections. 

6.2 Experimental Outcome 

We expected that the average post-test values across the 
experimental groups would be lower than the average 
post-test values across the control groups. The results 
contradict this expectation (Table 3) and indicate an 
opposite effect with mean post-test values of €1.06 and 
€0.98 for the experimental groups and only €0.74 and 
€0.85 for the control groups. After receiving the 
treatment, the average donation increased by 
approximately 30 cents to €1. This implies that our 
treatment - the passive player's Muslim immigrant 
background - had a positive effect on post-test values. 
As posited, no difference between the pre- and post-test 
values was observed in control group I, which ensures 
internal validity of the experiment. 

Due to these unexpected findings, we discussed 
methodological shortcomings in class and the students 
were encouraged to undertake further statistical analysis 
individually in order to investigate these counterintuitive 



findings. Students undertook one-way analysis of 
variance (ANOVA) and regression analysis, which found 
the difference between the experimental and control 
groups to be statistically significant (Table 4). Students 
who received prior statistical training presented the 
results to their fellow classmates and explained their 
interpretation. 


Table 3: Descriptive pre- and post-test values 


Items Groups 

N 

Mean 

S.D. 

EG-1 Pre-test 

41 

0.72 

0.49 

Post-test 

41 

1.06 

0.52 

Difference 

41 

0.33 

0.48 

CG-1 Pre-test 

38 

0.77 

0.56 

Post-test 

38 

0.74 

0.55 

Difference 

38 

0.00 

0.16 

EG-2 Post-test 

39 

0.98 

0.57 

CG-2 Post-test 

33 

0.85 

0.63 

Table 4: Regression Analysis of difference between pre- 
and post.test 

Variables 

Coefficient 


Treatment 

0.324** 




(0.083) 



Age 

-0.008 




(0.0145) 



Gender 

0.066 




(0.087) 



Fair-Trade 

0.050 




(0.044) 



Constant 

15.425 




(28.754) 



Adj. R 2 

0.158 



F 

4.66 



N 

79 




Note: Standard error in parentheses, **p<0.01 


In addition, students tried to explain the unexpected 
outcome by accounting for treatment groups and 
participants' characteristics such as age, gender, and 
attitudes towards fair-trade (Table 4). Recalling the 
previous sub-section, there was no stark variation among 
participants in terms of age, nationality, current 
residence, or place of origin. As a result, these variables 
had no significant impact on the difference between pre- 
and post-test values. Because these variables showed an 
insignificant impact and the treatment assignment was 
the only significant factor, the students had lively 
debates about the theoretical assumptions, stimuli 
design, and questionnaire design as possible pitfalls. 

The result rejects the hypothesis, but is in line with the 
theoretical sampling of the study: discriminatory 
attitudes were not detected among students in East 
Germany. As mentioned earlier, discriminatory attitudes 
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are least likely among our sample group. Hence, 
detecting discrimination would have been cause for 
great concern, whereas our finding confirms the 
assumption that students are less likely to discriminate 
against Muslims. This outcome also offers the positive 
prospect that students are unlikely to be negatively 
influenced by populist views, which were raised 
throughout Germany's immigration debate. This con¬ 
firms previous findings and contributes additional 
evidence that suggests the importance of education as 
the best way of preventing discriminatory attitudes. 

6.3 Possible Noises 

Due to the opposite outcome of what the students 
hypothesized, they were strongly engaged in a debate 
about possible pitfalls and threats to the internal validity 
that could have biased the results. The following issues 
were discussed. 

The aforementioned descriptive analysis of the 
participants' attitudes toward coffee and fair trade 
shows that players lean more toward favoring them. 
Even though the regression analysis indicated that 
attitudes towards fair-trade had no direct impact on the 
outcome, one can speculate that detailed information 
(i.e., the treatment) increases the general tendency to 
give more money to the businessman. 

Future experiments should have a control group in 
which the same information is presented, but without 
the photo and name of the entrepreneur. By doing so, 
participants will be more likely to imagine a similar 
entrepreneur but who belongs to the dominant social 
group. In addition, a third set of treatment and control 
groups could be introduced to present a different type of 
businessman and apply a typical cliche of the low-skilled 
entrepreneur. Moreover, a fourth possibility is to give 
only the names of the entrepreneurs and leave out any 
additional information about the business. Hence, those 
additional experimental and control groups are highly 
suitable to narrow down the causal link. 

Moreover, it is worth noting that in the 79 pre-tests, 
the majority (23 participants, 29 percent) chose to split 
the two euros equally. In the 151 post-tests, the majority 
(44 participants, 29 percent) also chose one euro for her- 
or himself and one euro as a donation to the 
businessman in Hamburg. This result could be a product 
of the Hawthorne effect: knowing they were being 
observed, the participants opted to give equal amounts 
of money to demonstrate fairness. The concern for the 
principal implementation gap is not only evident in self- 
reported surveys but also evident in experimental 
analysis. It may have been preferable for the final answer 
in the post-test (i.e., the amount of money the 
participant wishes to donate) to be given during the self- 
administered questionnaire in a sealed envelope to 
reduce the interviewer effect (Bryman 2008, p. 218), 
rather than having participants openly express their 
donation amounts, as was done in the current case. 


7 Conclusion: What We Have Learned from This Class 
Project 

7.1 Academic Lesson 

We expected students to exhibit more suspicion of 
Muslim immigrants, but found that they appear to 
exhibit no discriminatory behaviors toward Muslims. By 
and large, they act favorably or at least fairly toward 
Muslims. Hence, the findings rejected our stated 
research hypothesis and our theoretical assumptions 
about the student sample. Although the generalizability 
of the results is limited, students might, indeed, be less 
likely to show discriminatory tendencies. This leads us to 
further assume that the magnitude of discriminatory 
tendencies varies among social groups in Germany and is 
not distributed equally throughout the whole population. 

Further investigation and experimentation into German 
attitudes toward Muslims is strongly recommended. The 
question of what constitutes a better method for 
investigations remains an intriguing one. As we noted, 
surveys, particularly when people are directly asked for 
their opinions, are prone to bias, as the issue of social 
desirability enters into the equation. Furthermore, they 
can only be used to establish correlations - not causality. 
In order to establish a strong causal link, there is great 
merit in adding the element of experimentation, as we 
suggest has been demonstrated here. While the internal 
validity of this type of method may be high, greater costs 
and a lack of access to specific social groups to achieve a 
representative sample often make experiments difficult 
to introduce. 

Our investigation suggests that future survey research 
studies into in-group/out-group bias would greatly 
benefit from incorporating an experimental element. 
With the advent of the Internet and improved online 
surveying, there has been growing interest in survey 
experiments as part of research design (Gaines et al. 
2007; Mutz 2011; Sniderman 2011). The design of survey 
experiments consists mainly of a survey, with an 
experimental treatment (e.g., the biographical sketch of 
the businessman we used) added to the mix. These 
surveys can be easily conducted online and combine the 
advantages of both research designs, which is 
establishing high internal as well as external validity. This 
technique is being increasingly used in the fields of 
political science and sociology, and has even been 
applied to the study of attitudes surrounding the issue of 
immigration in the United States (USA) (Hainmueller, 
Hiscox 2010). Thus, such online survey experiments are 
recommended for future investigations into 
discriminatory behaviors toward Muslim immigrants in 
Germany, particularly because they can easily be 
conducted as part of a postgraduate course as described 
by Kam (2013). Due to budget constraints, such survey 
experiments can be undertaken with student 
convenience samples using university email systems as a 
recruiting tool, and software such as Qualtrics, Survey 
Monkey and LimeSurvey can be used as free 
demonstration copy in an experiment with a smaller 
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scope (Kam 2013, 9). To achieve a convenience sample 
that is not solely based on students, Amazon's 
Mechanical Turk has been used in postgraduate courses 
and recent studies (Kam 2013, 9; Berinsky et al. 2012). 
However, its application is limited to the USA and 
requires financial resources. 3 

Another recent development to achieve a wider 
convenience sample is the use of Facebook to advertise a 
study (Samuels, Zucco, 2013). This approach can be a 
relatively affordable means to recruit participants with 
various backgrounds and allows stratifying the sample 
(Samuels, Zucco, 2013, p. 12). Furthermore, free 
software (z-Tree) has been developed to conduct 
computer-assisted economic experiments more easily 
(Fischbacher 2007). Hence, technological developments 
will continue to facilitate the use of experiments in social 
sciences; therefore, students need methodological 
training in experimental research to adequately employ 
these tools. 

7.2 Pedagogical Lesson 

The lecturer assessed whether the two learning 
objectives were met primarily by the research papers 
that each student submitted at the end of the module. 
The lecturer was delighted to find that each student 
presented a well-structured scientific report. They 
recounted in detail what they had learned in the 
literature review, what they did in the experiment to 
collect the data, and further, offered statistical analysis 
of the data and commented on their findings. Each 
student also reflected on the limits of the project and 
indicated room for improvement. Due to the hands-on 
approach and active involvement, the students were 
empowered to take responsibility for the project and 
conducted the experiment with a high level of 
motivation. Moreover, they were encouraged to 
collaborate in the publication of the results, which is 
increasingly relevant for postgraduate students who 
aspire to a career in academia. 

It should also be mentioned that the lecturer had 
considered the students' prior experience of this subject 
when designing the module. From the process of 
designing the experiment to actually carrying it out, the 
lecturer observed that students were highly engaged in 
their project which showed that they had a sound 
perception of their learning situation. Encouraging the 
students to submit a fully-developed research paper also 
allowed the students to reconstruct and recount their 
learning experiences. Overall, the lecturer believed that 
the two main learning objectives were met. 

Given the limited time and financial resources of this 
class, the learning experience for students was 
comprehensive. Doing their own project improved the 
students' ability to understand and to analyze other 
scientific articles, methods, and research projects. 
Students believe that they learned a great deal about 
experimental methods in this module. Furthermore, the 
project improved their knowledge of the relationships 


between Muslim and non-Muslims in Germany and how 
they can answer questions in an empirical scientific 
manner. Involving students in all stages of a research 
project also increases their engagement and motivation. 
Because students had to consider and conquer each step 
from the conceptual design, data collection to the 
analysis in collaboration with their classmates, they were 
exposed to a wide range of possible pitfalls. Hence, this 
research experience has well-equipped the students for 
any kind of primary data analysis, which could include a 
thesis project or even a career as empirical social 
scientists. 

As for areas for improvement, the students and the 
lecturer agree that it might be better to have two 
modules in the future. The first would focus on teaching 
the theoretical underpinnings of the project, thus paving 
the way for a subsequent research seminar that would 
focus on planning and running the experiment. This 
would give students more time to learn and develop. In 
addition, this could enable the class to conduct a second 
experiment as cross-validation with students from 
another university or a different convenience sample 
that is not based on students. 
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Endnotes: 

1 We are aware that some recent studies have presented 
counter-evidence to in-group favoritism. For instance, 
Giith et al. (2005) found no significant signs of in-group 
favoritism in their German subjects. Yet, there are 
quantitatively more findings in support of the in-group 
favoritism theory. 

2 Various possible threats to internal validity and external 
validity as mentioned in academic textbooks (see Bryman 
2008, 38-39) were considered. However, due to the 
space limitations of this lesson report, we do not 
elaborate further. 

3 Therefore, we do not recommend it for universities 
which cannot offer financial grants to students. In the 
above mentioned case, each student received a research 
fund of $220 to collect data using Mturk at Vanderbuilt 
University (Kam 2013, 9). 
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